DENSITY OF VALUES OF LINEAR MAPS ON QUADRATIC SURFACES. 
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Abstract. In this paper we investigate the distribution of the set of values of a linear map at integer 
points on a quadratic surface. In particular wc show that this set is dense in the range of the linear 
map subject to certain algebraic conditions on the linear map and the quadratic form that defines 
the surface. The proof uses Ratner's Theorem on orbit closures of unipotcnt subgroups acting on 
homogeneous spaces. 



1. Introduction 
We are motivated by the following general problem. 

Problem 1.1. If X is some rational surface in and : X ^ M'* is a polynomial map, then what 
can one say about the distribution of the set {F (x) : x ^ X Ci Z''} in W 1 

In full generality problem 11.11 is unapproachable via available techniques, however if X has a large 
group of symmetries then the problem can be studied from a dynamical systems point of view. The 
main result of this paper deals with a special case of the above problem and is stated in the following 
Theorem. 

Theorem 1.2. Suppose Q is a quadratic form on Mf^ such that Q is non-degenerate, indefinite with 
rational coefficients and signature {p,q)- For a G Q \ {0} define = G R'^ : Q {x) = a} and 
Xz = {x & Z'^ : Q {x) ~ a}, suppose that Xz is non empty. Let M : — >■ R'* he a linear map such 
that 

(1) The following inequalities hold, d > 2s and rank{Q\M=o) > 2. 

(2) The quadratic form Q\m=o "is indefinite. 

(3) For a/Z a G R'" \ {(0, . . . , 0)}, aM is non rational. 



Then M{Xz) 

Density of the set in problem 11.11 has been established in some special cases. Theorem 11.21 gives 
conditions sufficient for density in the situation described in the Theorem. The simplest instance of 
problem 11.11 is when Xz = Z'^ and _F : R'^ — ^ R** is a linear map, in this case we have F {Xz) = R'' 
provided that s < d and condition [3] of Theorem 11.21 is satisfied for the map F. This result can be 
proved using elementary methods, see |Ca57j . Theorem 1, page 64, for example. 

The case where X^ = Z'' and is a quadratic form is dealt with by the so called Oppenheim conjecture 

which says, provided that Q is a non degenerate, indefinite quadratic form in d ^ 3 variables such 

that Q is not a multiple of a rational quadratic form, then Q (Z'') = R. After 50 years of unsuccessful 

attempts to prove the conjecture using methods from analytic number theory (see |Ma97| for a survey 
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of partial results obtained using these methods), it was finally proved in the late 80's by G.A. Margulis 
in [Ma89| . His method of proof used a surprising connection, first noted by Ranghunathan, between 
the orbits of SO (Q) in the space SL^ (R) /SL3 (Z) and the Oppenheim conjecture. In particular, what 
Ranghunathan noticed was that if you could show that SO (Q) SL-s (Z) was dense in SL-s (M), then 
this would be enough to prove the Oppenheim conjecture. 

Since the successful proof of the Oppenheim conjecture similar lines of reasoning have been used by 
Dani and Margulis in [DM90j to show that, for a pair {Q,L) , consisting of a non degenerate quadratic 
form, Q, and a non-zero linear form, L, the set {(Q (x) , L {x)) : x G Z"^} is dense in provided for 
all a, /3 S M \ {0}, aQ + /3i^ is not rational and the plane given by {x S : L{x) — O} is tangent to 
the surface given by {a; e R3 : Q(a;) = 0}. This result was later extended by A. Gorodnik in [Go04| 
who showed that if (Q, L) is a pair as before, in dimension d ^ 4, such that Q\l=o is indefinite and for 
all a, /3 e R \ {0}, aQ + filF' is not rational, then { (Q (.t) ,L{x)) : x £ Z"^] is dense in R^. Recently in 
|DS08| Dani and Shrikrishna prove what is in some sense an extension of the above work of Gorodnik. 
They prove density for a system consisting of d — 2 linear forms and a quadratic form under similar 
algebraic conditions as in |Go04j . The major difference is that the result of |DS08| applies only to 
almost all linear forms. 

The key fact that enables us to study the above examples is that the group of isometrics of the maps 
in question are large. Moreover, except in the latter example they are generated by one parameter 
unipotent subgroups and it is the absence of this property that is responsible for the result of |DS08| 
only being applicable almost everywhere. 

We now make some remarks concerning the conditions of Theorem 11.21 

Remark 1.3. There is no reason to expect that the inequalities of condition [1] arc optimal, however 
they are necessary for the proof to work. It is reasonable to expect some inequalities of this type 
to be necessary. The condition that rank (Qj a/^q) > 2 is analogous to the condition that d > 2 in 
the Oppenheim conjecture and as such we believe this condition to be necessary, although no counter 
examples have been found to indicate this. 

Remark 1.4. Condition[2]is possibly stronger than is strictly necessary, however it is a natural condition 
and comparable with conditions imposed in [Go04] . It implies the necessary condition that the set 
Ci {x G : M (x) = &}, for some 6 £ R" is non compact. To see that this condition is necessary, 
suppose XRn{x e R"* : M (x) ~ 6} is compact. Hence XRHija; £ R'' : \AI (x) — 6| ^ e} is also compact 
and therefore contains only finitely many integer points. Hence if 6 ^ Z*, we can make e small enough 
so that Xr n {a; e R'' : \M (x) — b\ ^ e} contains no integer points, but then there exists an open set 
Be (&) C R^ such that there is no x e Xz with M (x) e (6). 

Remark 1.5. Condition [3] is necessary since otherwise M (Z'') would not even be dense in W. 

2. Set up 

2.1. Construction of a dynamical system. The strategy we will use to prove Theorem 11.21 is 
analogous to that used by Margulis to prove the Oppenheim conjecture. We will use the following 
Theorem of M. Ratner found in |Ra94| . 
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Theorem 2.1. (Ranghunathan's topological conjecture) Let G be a connected Lie group and U a 
subgroup of G generated by one parameter unipotent subgroups, given a lattice T of G and any x G G/F, 
the closure of the orbit U x is equal to the orbit of a closed connected subgroup F such that U ^ F ^ G. 

In order to make use of Theorem 12.11 we need to construct a dynamical system, so define Gr = 
{g e SLd (M) : Q (gx) ~ Q (x)}, and let G = Gr be the connected component containing the identity 
of Gk. Let r G n SLd (Z) and H = {geG:M (gx) = M {x)}. 

It is a standard fact that G = SO (p, q)° is a connected Lie group. Since a priori, H may not be 
generated by one parameter unipotent subgroups, our first aim is to define H* < H such that H* is 
generated by unipotent subgroups, we will then consider the dynamical system that arises from H* 
acting on G/T. Note that condition [2] of Theorem 11.21 implies that H will be non compact, and so 
there is hope that such an H* exists, in section [2.31 an explicit description of H* is given. 
It is known that F is a lattice in G so long as G is non-compact and defined over the rationals, which 
happens when p,q ^ I and Q is a rational form. In particular both of these conditions follow from 
assumptions of Theorem [TT21 so in our case F is a lattice in G. 

2.2. A canonical form for the system. We would like to use linear transformations to trans- 
form our system (Q,M) into something more manageable. For two pairs {Qi,Mi) and ((32,-^2) 
wc say {Qi,Mi) ^ (Q2,Af2) if and only if there exists gd G GL^ (M) and gs G GLg (R) such that 

(Qi (x) , A'h (x)) = (O2 {9dx) , gsM2 (gdx)). 

The following result, adapted from |Go04| . is reproduced below for completeness and will be used to 
establish a more general form. 

Lemma 2.2. Every pair {Q,L), where Q is a non degenerate quadratic form on with signature 
{p, q), and L is a non zero linear form on , is equivalent to one and only one of the following pairs: 

(1) If rank{Q\L^Q) = — 1 then either, 

(a) {Q,L)^{Y.Uxj-Y.U+,xlx,) 

(h) (0,i)-(ELi^f-Et,+i^?,^/)- 

(2) If rank{Q\L^Q) = d - 2 then (Q, L) ^ {2xiXd + ^1 ~ Etp+i ^l^i) ■ 

Proof. By Sylvester's Law we can always transform Q ~ X]r=i ~ SiLp+i x^- Next by applying an 
element of SO {p, q) to the system it is possible to ensure that the coefiicient of Xi in L{x) is non zero. 
Now use the transformation xi — > xi -f L' (a;) for L' a linear form in the remaining variables, to get 
that 

(Q, L) ^ {x\ + xiL" [x) + Q' [x] , xi) 

for Q' and L" a quadratic form and linear form respectively, in the variables not including xi. Note 
that Q' has signature (p — \,q) or (p, 9 — 1) if rank (Q|l=o) ~ d — \ and signature (p — 1, g — 1) if 
rank((5|i=o) = d—2. Suppose we arc in the first case, apply a transformation in the variables not 
including xi, to get that 

(d p d \ 

xi + 2^aiXiXi+^x1 - ^ a;^,a;i , 
i=2 i=2 i=p+l I 
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where p = poT:p = p+ l. Next, we use transformations of the form 

Xk — akXi for 2 ^ k ^ p 



Xk 



Xk + ctkXi for p + 1 ^ /c ^ d 



to get 



1=2 i=p+l 1 4=2 4=33+1 

If the coefficient of x\ is positive, then p = p and we see that we are in case la of the Lemma, similarly 
if the coefficient of x\ is negative, then p = p + \ and we see that we are in case lb of the Lemma, 
after relabeling. 

Suppose that rank(Q|L=o) = d — 2, apply a transformation in the variables not including xi to get 
that 

(d p d-1 

xj + 2'Y'=^iXiXi +^,^1 - E 
i=2 i=2 i=p+l 

Next, for 2^A;^ff— Iwe use transformations of the form Xk — > ± akXi to make ai zero for all 
i ^ d. Note that ad ^ 0, otherwise Q would be degenerate, so to finish off we use the transformation 
Xd {xd — xi) and we see that we are in the second case of the Lemma. □ 



We can now prove the main Lemma of this section. 

Lemma 2.3. For any pair (Q, AI), where Q is a non- degenerate quadratic form on M** with signature 
{p,q), and M : — > W is a linear map, if Q\m=o is indefinite then {Q,AI) ^ {Qo,AIo), where 

m s+r s+r+Ti 

Qo{x) =Q,n+l,...,s{x) +2Y^i^d-t+l+ E ^» ~ E 

2—1 Z— S+1 2— s+r+1 

Mo {x) = {xi, . . . ,Xs) 

and m ~ d — s — rank{Q\i\j=o) o.'^^d Qm+i....,s (x) is a non degenerate quadratic form in variables 
Xm+ii ■ ■ ■ ,Xs with signature (p' , q'), such that the following relations hold, r ~ p — m ~ p' ^ 1 and 
n = q — m — q'^1. 



Proof. We proceed by induction on s. For s = 1 we know from Lemma 12.21 that the conclusion of the 
Lemma holds, so suppose the Lemma holds for s ^ fc — 1. Let s — k, and suppose that M = (Li, . . . , Lk) 
for Li, . . . , Lfe non zero linear forms on R'*. 

If rank (QIlj^o) = d — I, using Lemma 12.21 it is clear that we can transform our system into 

Mj^y ixi,L'^,...,L',) J' 

where Xj, . . . , LJc ^^'^ linear forms on M'' and / G {1, d}. Next we can eliminate the coefficient of xi 
in . . . , L'j, by subtracting some multiple of xi, then relabel xi — ?> xi and xi — )■ xi and apply the 
inductive hypothesis to see that the conclusion of the Lemma holds. 
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If rank {Q\li=o) = — 2, using Lemma [2T2l we get 




2™ + EL2-?-Et' 



2 -^i /-^i=p+l 



X; 



(xi, L2, . . . , LJ,.) j 

Again we can eliminate the coefficient of xi in Lji ■ • • i ^/c by subtracting some multiple of xi. Suppose 
that the coefficient of Xd is zero for each L2,...,LJ,, in this case we are in position to apply the 
inductive hypothesis and get to the conclusion of the lemma. Suppose the coefficient of Xd in L[ is 
non zero for some 2 ^ i ^ fc, without loss of generality suppose that i = 2, in particular suppose 
L'2 {x) = L" (x) + cidXd for some linear form L" in variables X2, . . . , Xd-i- Use a transformation of the 
form Xd ^ {xd — L" [x)) to get that 



^^XiXd + X]i=2 ^iPi^l + X]i=2 



X, 




\ M J y {xi,Xd,L'^, ■■■,L'f,) J 

Next, for 2 ^ k ^ d — 1 we use transformations of the form Xk — > x^ ± f^^xi to make f3i zero for all i. 
After we have done this we end up with 

V {xi,Xd,L'^,...,L'f^) ) 

so if we relabel X2 — > Xd and Xd — > X2 and eliminate these co-ordinates from Lg, . . . , we see we can 

apply the inductive hypothesis and get the desired conclusion. The assertion that Qm+i s {x) is a 

non degenerate quadratic form in variables x^+i, ■ ■ ■ ,Xs follows from the fact that Q is non degenerate. 
We see that Qm+i....,s {x) has signature {p' , q') where r ~ p — m — p' and n = q — m — q' because the 
signature of 2 xiXd-i+i + Ylritl+i ~ J2i=l+r+i ^1 + m^n + m). Finally, the assumption 
that Q\m=o is indefinite means that r ^ 1 and 1. □ 



Define gd 6 GLd (M) and gs G GLs (M) such that we have Q {x) = Qo {gdx) and M (x) = g^Mo (^^a;)- 
The conjugate dynamical system consisting of H acting on G/T is defined by relation gdGg^^ ~ G, 
and the corresponding relation for subgroups of G. Note that for all 5 G G we have Qo (gx) — Qo (x) 
and for all h & H we have Qo (hx) — Qo {x) and Mo (hx) — Mo {x). 



2.3. Definition of H*. We will use the notation Ip to denote the p x p identity matrix and Ip q to 
denote the indefinite identity matrix with signature {p,q). For 1 ^ i ^ m and ti G M''^'*^'™ we define 
the linear transformations 



u: 



Xj + tijXi 



for j ^ s and j > d — m, j =/= d — 1 + i 
for s + l^j^d — m 



Xd—l-{-i ^ 1+i ^r.7i^iX ^r,n^iXi for X 



s+1: 



■ ; Xd—rn ) 



One can check that H contains the subgroups Ui — {t/j. : G R'' * ™} and the subgroup Do, 0,0 
defined as follows 



/ -^s-ii-i2— is 



D,. 



so (r + zi + Z3, 71 + Z2 + 13)° 
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for convenience we denote Uq.o.o ~ D, so we see that 

H* = Ui... UrnD H. 

One sees that H* defined in this way is connected and generated by one parameter unipotcnt subgroups 
since the Ui ^s are themselves unipotcnt subgroups and conditions [1] and El of Theorem 11.21 imply that 
D is generated by one parameter unipotcnt subgroups. To see this note that rank ((5|m=o) > 2 implies 
that r + n ^ 3. Moreover, as noted in Lemma 12.31 the fact that Q\m=q is indefinite implies that r ^ 1 
and n ^ 1. 

3. Lemmas concerning subspaces invariant under the action of subgroups of SO(jj,q). 

We wish to obtain information about the subgroup F such that H* ^ F ^ G and for any x G 
G/F the orbit H*x — Fx. First we introduce some notation, let C be the space of d dimen- 
sional linear forms defined over K and M = {Li, . . . , Ls) for £ C. For any group G let C-^ = 
{L <E C : L {gx) ~ L (x) for all g G G} be the 'fixed vectors of G". 

In particular we are interested in subspaces of C invariant under F , the main work of this section is to 
classify such subspaces. The first step is to show that there are no invariant rational subspaces in the 
fixed vectors of H* , this is then extended to algebraic subspaces and finally to any subspace defined 
over C. 

First we prove that the set of vectors fixed by H* is exactly the set of linear forms that make up M. 
We will use the notation £,„ = (x^+i, . . . , Xd-m)- 

Lemma 3.1. C"' = {Li, ...,Ls). 

Proof. It is clear that (Li, . . . , Ls) C . So suppose (Li, . . . , Ls) £ . This means there exists 
T E such that T ^ (Li, . . . , Ls) , this is equivalent to saying, there exists T £ £^*such that 
T ^ (xi, . . . , Xs). Since T ^ {xi, . . . , Xs), if for t eM.'^ we write T {x) = tx, we can suppose ti, . . . ,ts = 0. 
Furthermore since Z),„ < H*, we may suppose that is+i, ■ • ■ ,td-m = since 5*0 (r, n)° has no fixed 
vectors. But we also have Ui . . . Um < H* and the space {xd-m+i, ■ • ■ , Xd) is clearly not Ui . . . Um 
invariant. This implies that td-m+i-, ■ • ■ , = 0, and thus T = and wc have a contradiction. □ 

Recall, condition [3] of Theorem 1 1.21 is that for all a e R'* \ {(0, . . . , 0)}, aM is non rational. From this 
and Lemma l3.1l we can deduce the following. 

Corollary 3.2. There exists no non trivial F invariant subspaces defined over Q and contained in 

Proof. Suppose there exists U C such that U is defined over Q, or in other words, U has a rational 
basis. By lemma 13.11 it is clear U C (i^, . . . ,Ls) ■ Now let ui, . . . , Uq be a basis for U, then by the 
preceding remark, we can write Ui as a linear combination of the Lj 's. Since we are supposing Ui is 
rational, this is in contradiction with condition [3] of Theorem II .21 since this condition implies that no 
linear combination of Li is rational. □ 
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We continue the process of classifying F invariant subspaccs of C. Since D ^ F any F invariant sub- 
space will be D invariant, it is for this reason the next Lemma, which classifies two distinct possibilities 
for any D invariant subspace, will be useful. 

Lemma 3.3. If V Q C is a D invariant subspace then either 

(1) V 

(2) V ^ Cm®U where U C . 

Proof. First note that if C then it is clear that V will be D invariant. Suppose that V C £ 
is a _D invariant linear subspace such that V ^ , since C = ® €,„. there exists v & V such 
that V ^ vi + V2 for vi £ and V2 G Cm, with V2 0. Now for any d € D we have dv ^ v = 
vi + dv2 — {I'l + W2) = dv2 — V2 & V, since consists of forms fixed by D. We can choose c? e D so 
that w = dv2 — U2 7^ 0, but w ^ V Cl Cm, this means that (Dw) is a D invariant subspace such that 
(Dw) C Cm, but this implies (Dw) = Cm. because D acts irreducibly on Cm- Since (Dw) = Cm., we 
have that Cm ^ V and as w = vi + V2 & V for vi G C^ and V2 G Cm we have vi = v — V2 G V and so 
we see that V ~ Cm (B {V n C^), which implies that we are in the second case. □ 

For the same reasons as before it will be useful to classify distinct possibilities for any H* in- 

/ B„ 

variant subspace of C. We will use the following notation Jm = 



Bm. 



V 1 



1\ 



Id-2m 
V Bm 



where 



G Mat,, 



Lemma 3.4. IfV^Cisa H* invariant subspace then either 

(IjVCC"^ _ 
(2) V = JmCo ® U where U C J,„£^* . 

Proof. First note that if F C then it is clear that V will be H* invariant. Suppose that V C C is 
an H* invariant linear subspace such that V ^ C^' , since D ^ H* it is clear V will be D invariant, 
thus by Lemma 13.31 either 

(1) VCC^ 

(2) V = Cm®U where U C C^ . 

If we are in case (1), then since C^ = C^* © (xd-m+i, ■ ■ ■ , Xd) we can suppose there exists v &V such 
that V = vi + V2 for vi G and V2 G (ccd-m+i, • ■ • ,Xd)- Suppose that V2 ^ 0, then there exists 
u e Ui . . . U.m < H* such that uv2 ^ C^ and hence uv2 ^ V, but is a contradiction since V is supposed 
to be H* invariant. 

If we are in case (2), from the definitions oiUi . . . Um, Cm and Jm we see that (Ui . . . UmCm) ~ {JmCo), 
then since Ui . . . Um < H* , the fact that Cm ^ V and V is H* invariant implies that V = JmCo + U 
for some U C C. Moreover, since C = JmC^' © JmC^ we see that V = JmXo ® {v r\ JmC^'^ which 
implies we are in the second case of the Lemma. □ 
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We can reformulate the above as follows. 

Corollary 3.5. If V <^ C is a H* invariant subspace then either 

(1) V C C"' 

(2) V = gJJ,nCo © U where U C gJj,nC"' . 

Proof. Note that if V <^ C is a H* invariant subspace, then g^^V is a H* invariant subspace, and 
that gJC"' = C"^ . a 

The next Lemmas are central to the arguments in the next section. We use Q to denote the algebraic 
closure of Q. 

Remark 3.6. A key fact that will be used in the following will be Proposition 3.2 in |Sh91j which 
says that F ^ F (M)°, where F is an algebraic group, defined over Q and that the radical of F is 
a unipotcnt algebraic group defined over Q. In particular this means F D SLd (Q) = F. This will 
be used in conjunction with the Levi decomposition F = FrF^ where Fr is reductive and F^ is the 
unipotcnt radical, see |OV90| . chapter 6, page 282, for details. In particular we see that Fr is in fact 
scmisimple and Fr can be chosen to be defined over Q. 



The following Lemma and its Corollary can be seen as refinements of Corollary [ 

Lemma 3.7. There exists no non trivial F invariant suhspaces defined over Q and contained in . 

Proof. Suppose for a contradiction there exists at least one non trivial F invariant subspace defined 
over Q and contained in . Define V to be the unique maximal F invariant subspace such that 
V C and V is defined over Q. For a G Gal(Q/Q), we write (J {V) to mean the vector space 
spanned by the vectors that arise from applying a to all components of all basis vectors of V . It 
follows that for any a G Gal (Q/Q) we have that (t (V) is F n SLd (Q) invariant. By remark 



we have F D SLd (Q) ~ F, and therefore a {V) is F invariant. Additionally, this means that a {V) 
is H* invariant, hence Corollary 13.51 implies, either a {V) C , or a {V) — gJJmCo © U where 
U C gJj^C^* . But dim{a{V)) < s < d — s < dim [gj JmjCo (B U) and so we must have that 
cr {V) C . By definition of V, we see that a {V) C V, and thus <t {V) = by considering 
dimensions. By a Proposition on page 30 of |Bo91] this means that V has a rational basis, contradicting 
Corollary [XI □ 

Corollary 3.8. There exists no non trivial F invariant suhspaces defined over Q and contained in 

Proof. Lemma l3 . 71 implies that if is an invariant subspace defined over Q and contained in gj 
then V is not contained in £^ . But since V should also be H* invariant Corollary 13.51 implies the 
only other option is that V = gJJmCo © U where U C gJJmC^' ■ This is impossible since in this case 
V would not be contained in gJC^ = © {xd~m+i, ■ ■ ■ , Xd). □ 



We can now use CoroUarv 13 . 81 together with results of Shah detailed in remark [Xdl to establish that F 
is semisimple and to further classify F invariant suhspaces of C. 

Lemma 3.9. F is semisimple. 
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Proof. We prove that C^^ = C, this implies that the unipotent radical is trivial and thus by remark [3?^ 
we will be done. Since is Fr invariant and Fr is reductive there exists an F^ invariant complement, 
W, to C^" such that C ~ C^" © W . Let S = W or C^" , note that since Fr can be defined over Q in 
either case S can be defined over Q. Now since D ^ F its clear that gdDg^^ ^ F. Moreover since D is 
reductive, by Malcev's Theorem, found in |OV90j . Theorem 5, page 287, there exists u G Fu such that 

1 X 1 X 

u" gdDg2 u ^ Fr. This means that S is gdDg^ u invariant and hence g^ u S is D invariant, 
and thus by Lemma either vT' S C g^C^ , or u'^S = gj^m ® U where U C gj£^. Moreover, by 
comments on page 10 of |Bo66| we see that u e F„ n SLd (Q) and so u^S is defined over Q, thus, 
CoroUarv 13.81 implies that u^S = gj^m ® U where U C gJC^. This is a contradiction since u'^C^" 
and u^W should have an empty intersection. This shows us either ~ or W = 0. Finally we 
note that since Fu is unipotent C^" ^ 0, this proves the Lemma. □ 

Lemma 3.10. There exists no non trivial F invariant subspaces defined over Q and contained in C. 



Proof. By Lemma 13.91 F is semisimple and hence completely reducible. Suppose for a contradiction 
that C = Vi ® . . . (B Vk where each Vi is a. non trivial, irreducible F invariant subspace. By remark 
13.61 F can be defined over Q, this implies that each of the Vi can be chosen to be defined over Q. 
Because H* < F each Vi is H* invariant. Thus by Corollary 13.51 all but one of the spaces Vi are 
contained in . Without loss of generality let W — Vi ® . . . (B Vk~i C , but since each V is 
F invariant so is W^. It follows that we have found an F invariant subspace defined over Q contained 
in in contradiction with Lemma 13.71 Thus we have a contradiction as required and our original 
decomposition of C must have been trivial. □ 



The following is the final Lemma of the section and completes our classification of F invariant subspaces 
of C In the course of the proof the notations of weights and weight spaces are used, the reader is 
directed to numerous books on Lie groups for details of this subject, for instance |OV90| . 

Lemma 3.11. There exists no non trivial F invariant subspaces defined over C and contained in C. 



Proof. Let f denote the Lie algebra of F , t a Cartan subalgebra of f and t* its dual. By Lemma 
f is semisimple. We use the language and notation of Serre, [Se87| . Lemma [3.101 savs that, if is 
the space of linear forms defined over Q then Cq is an irreducible f-modulc. Thus the Theorem of the 
highest weight, which can be found on page 60 of [Se87| . implies that £q has a highest weight. If Cc is 
the space of linear forms defined over C, we claim that the weights of Cc are the same as the weights 
of It is easy to see that the weights of £q form a subset of the weights of Cc- The situation 
when there is a weight of Cc, that is not a weight of C^, is the one we should rule out. Since Q and 
C are algebraically closed wc can decompose C^ = C^ and Cc — ®tjet* ^C- '^^^ write 

Cc = >Cq(2)qC, and thus combined with our decomposition for C^ we get that Cc = ©^^gf '^^®q 
Comparing this with the earlier decomposition of Cc shows us for any weight, w, we have £^(8)qC = C^. 
Thus, if C^ = 0, then = 0, this implies the claim. Therefore Cc has a highest weight and thus Cc 
is an irreducible f-module, this implies the Lemma. □ 
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4. Proof of the main Theorem. 

To prove Theorem II .21 wG will proceed to show that the group F obtained from the application of Rat- 
ner's Theorem is G ~ SO {p, q)°, once this is established Theorem 11.21 follows by standard arguments. 
Since, in the course of proving this fact some cumbersome notation is used, possibly obscuring the 
underlying idea, an outline of the proof is presented as follows. 

(1) The Lie algebra of F is decomposed into subspaces defined in terms of 4 by 4 block matrices. 

(2) Then it is shown that, if the intersection of these subspaces with the Lie algebra of F is trivial in 
certain cases, then F will have non trivial invariant subspaces contained in £^ , contradicting 
Lemma 13.111 

(3) Therefore the intersection of these subspaces with the Lie algebra of F is non trivial, this is 
used to show that F contains an enlarged copy of H*. 

(4) This process can then be repeated until it is shown that SO {p, q)° ^ F. 

We now proceed with the actual proof. 
Lemma 4.1. F = SO (p, q)° . 

Proof. We look at the Lie algebra of F^ denoted f. Since H* = Ui . . . UmD ^ F the Lie algebra of H* , 
denoted f)*, is a subalgebra of f. The aim is to show that this implies one of the following, 

(1) C/i . . . C/™i?i,o,o ^ F, 

(2) C/i . . . U,nDo.ifi ^ F, 

(3) C/i . . . U,n~iDo,o,i ^ F. 

The first two cases occur when r + n < d — 2m and the third case occurs when r + n = d — 2m. Once 
we have established the above claim, then we can replace {r,n,m) by (r + l,n,m) in the first case, 
by (r, n + 1, m) in the second case and by (r + 1, n + 1, m — 1) in the last case. Then it is possible to 
repeat the entire argument until the claim of the Lemma has been obtained. 

Let 

^ fii hi hi fii ^ 

J _ .fl2 1-22 h2 fl2 
fl3 h3 hs /43 
\ /l4 hi hi /44 / 

where /n, /i4, /41, /44 e Mat„, (K), > /21 , /J2 > /24 e Mat,_„i,„ (R), h2 e Mat,_,„ (M), ha e 
Matd-s-m(K), h2,f23 G Matd-s-m,s-m (K) and hi, fIsJsA, fTs e Matd-s-rn,m (K)- Since f is a 
subalgebra of so (Qo) we have that any / G f must satisfy the relation f^Q' + Q' f = 0, where Q' is 
the matrix that defines the quadratic form Qq. So, suppose that / G f, and compute as follows. 
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where Q" is the matrix that defines Qm+i,...,s- The above computation yields the following, /14, /41 S 

SO{Bm), /22 e SO(Q"), /33 £ SO{r,n), /12Q" = -Bmf2i, fl3.Ir,n = ^Bmf'^i, /ll^m = -BmfJi, 
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following subspaces 



-0'7J2 and hiBr, 
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Since D ^ F, the Lie algebra m is a subalgebra of f. Therefore, considering f, m, and a as vector 
spaces, we have that m © ((d © o) n f) C f. 
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Let Ofe = {t G such that Sij = for all j ^ k}, in other words, Dfc is the subspace of that consists 
of the fc*'' row of D. Note that Dfc is a Lie algebra but is not unless s — m — 1. Again by considering 
the Ofc's and as vector spaces we can write = Di © . . . © Os-m where we cannot split the O^'s into 
further m invariant subalgebras since [h, v] e Vk for all h G m and w £ 0^-. So to summarise the above, 

m © ((Oi © ... © Os~m © a) n f) c f. 

Since f is a Lie algebra and hence closed under the Lie bracket, we have the following implication, if 
f n 7^ 0, then f fl 7^ , this follows since [)* is a subalgebra of f and ti* = u~ © m, then choosing 
/i G f)* and w G 0^ we see that [h, v] n contains non trivial elements. Additionally, we note that if 
r + n < d — 2m, then the above observations imply that f n 7^ 0, since otherwise f would be contained 
in a subspace of the form 
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/ 



This would imply that there exists an F invariant subspace V C contradicting Lemma 13.111 

Therefore at least one of the 7^ 0, so without loss of generality suppose that Vg-m 7^ 0. This means 
that 

f D m © ((Ui © ... © Vs-m © a) n f) 
= m © Os-,n © ((Oi © ... © o^-m-i © a) n f) 

{so (r, n + 1) © ((Di © ... © Vs-rn-i © a) n f) if q' ^ I 
so {r - 1, n) © ((Oi © ... © Vs-m-i © a) n f) otherwise, 
where in the last step we use that Vs-m 7^ and the following relation 

{so {r + l,n) for 1 ^ fc ^ p' 
so {r,n + 1) for q' < k s — m 

which holds provided that 0^ 7^ 0. So in the case that r + n < d — 2m, we have shown that either, 
C/i . . . UmDifl^o ^ForUi... t/™I?o,i,o F. 

We are left with the case when r + n = d — 2m. In this case the subspaces, a,ti^ and D become 
trivial and we arc really dealing with 3 by 3 matrices. For this case we introduce the additional 
notation. Let O = {d G O such that dik = for Ik 7^ ii} be the subspace of d consisting of elements 
of Z) such that every entry is zero expect for the entry in the jj*^ matrix position. Moreover, let 
= {u £ such that tu = for / 7^ fc} be the subspace of which consists of the k*^ row of u^. 

Considering and the u^fc's as vector spaces, we can write = © . . . © u^^ . Note that the 
u^fc's are Lie algebras and that we cannot split the u^^-'s into further m invariant subalgebras since, 
as the action of m on u^fc is irreducible we have that [s,u] G u^fc for all s G m and w G u^fe . Since 
C/i . . . UmD ^ F we have u~ © m C f and that 

f 2 u" © m © ((t) © u+ © b+) n f) , 

where again we consider the decomposition as a decomposition of vector spaces. 
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If f n b+ ^ 0, then f nu+ ^ 0, since f is closed under the Lie bracket. Exphcitly, f contains ()* = u~ ©m, 
and so taking b G b'^ and /i G [}* and checking that [u, b] nu+ contains non trivial elements verifies the 
assertion. 

This observation and Lemma l3. 1 f I means at least one of the is not zero since otherwise f would be 
contained in a subspace of the form 

/ * 

* * 

* * * 

which would mean there was an F invariant subspace V such that V C . Suppose without loss of 
generality that u+ ^ 0, since f is closed under the Lie bracket this implies that f flD ^ 0, where we 
can check this claim as before. Now we can compute 

f D u" ©m© ((c)©u+) nf) 

m—1 



V 




3 ©so (r + 1,71 + 1). 



i=l 



In the second step we use that t) D ^ and in the last step we use that u„j © m © O © u+ = 
so {r + l,n + 1), provided that u~ , u+ and arc not zero. 

This means that Ui . . . J7„i_iI?o,o.i *s F and we are done by the initial remarks. □ 
We are now ready to prove Theorem II. 21 



Proof of Theorem 1.2. Rewrite M {Xz) = {M (x) : x & Xz} and by using the definition of H* we see 
{M (x) : x e Xz} = {^jf {H*x) : x G Xz}. Now we can restrict our attention to a F orbit in Xz to 
get {M {H*x) : x £ Xz} ^ {M {H*rx) : x G Xz} . By Lemma 14.11 and Ratner's Theorem we have 
{M {H*Tx) : X G Xz} ^ {M {Gx) : x G Xz} and since G, being the identity component of SO {p, q), 
acts transitively on connected components of we have {M (Gx) : x G Xz} ~ {M (x) : x G Xr} since 
if Xr is not connected, then if a; G Xz we have —x G Xz and x and —x lie in the two separate compo- 
nents of Xr. The fact that Q is non-degenerate and indefinite means that Xr n {.t G M'' : M {x) =6} 
is non empty for every b G R'* or, in other words, that {M (cc) : a; G Xr} = W and so we are done. □ 

Acknowledgments. I would like to thank my supervisor, Alex Gorodnik, for suggesting the problem 
to me and for many helpful discussions and suggestions, generally ensuring accuracy of the stated 
results. 
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